Skip to content

DEP-277: Add CHEFS + Object storage file upload functionality to DEP (+fix survey listing) - #2892

Merged
NatSquared merged 6 commits into
mainfrom
DEP-277-formio-fileupload-backend
Jul 9, 2026
Merged

DEP-277: Add CHEFS + Object storage file upload functionality to DEP (+fix survey listing)#2892
NatSquared merged 6 commits into
mainfrom
DEP-277-formio-fileupload-backend

Conversation

@NatSquared

Copy link
Copy Markdown
Collaborator

Issue #: 🎟️ DEP-277

Description of changes:

  • Feature Added ability to upload files to Object Storage during survey submission
    • Added a new public document endpoint to the API that generates pre-signed URLs for uploading files to Object Storage. This endpoint takes the file name and content type as input and returns a pre-signed URL that can be used to upload the file directly to Object Storage.
      • The new API can also be used to generate pre-signed URLs for downloading files from Object Storage, as well as deleting files from Object Storage.
    • Updated the Formio options to include callbacks for handling file uploads, downloads, and deletions. These callbacks use the new API to generate URLs and perform privileged operations on Object Storage, allowing users to upload files during survey submission without needing high-level access to the storage service.
    • Cleaned up survey-related component code
    • Updated the survey pages to use React Router loaders to fetch the survey data instead of using useState and useEffect, as well as making sure the loader returns promises (as opposed to awaiting the data) so that the survey page can be rendered while the data is being fetched. This improves the user experience by allowing the page to render faster and show a loading state while the survey data is being retrieved.
    • Fixed authorization bug that did not allow Super Admins to view surveys

User Guide update ticket (if applicable):

    • Yes, a user guide update ticket has been created. (Link to ticket)
    • No, a user guide update ticket is not required.

Common component changes:

    • Yes, I have updated CONTRIBUTING.md and the docblocks to document any changes to the common components.
    • No, there are no changes to the common components.

@codecov-commenter

codecov-commenter commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 46.79245% with 141 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.30%. Comparing base (a8980ab) to head (ae31e02).

Files with missing lines Patch % Lines
api/src/api/resources/document.py 22.61% 65 Missing ⚠️
api/src/api/services/object_storage_service.py 27.90% 31 Missing ⚠️
web/src/services/objectStorageService/index.ts 26.66% 22 Missing ⚠️
api/src/api/services/survey_service.py 72.72% 9 Missing ⚠️
...eb/src/components/common/Navigation/Breadcrumb.tsx 36.36% 7 Missing ⚠️
api/src/api/services/authorization.py 81.25% 3 Missing ⚠️
...components/comments/admin/review/CommentReview.tsx 60.00% 2 Missing ⚠️
api/src/api/services/comment_service.py 75.00% 1 Missing ⚠️
web/src/routes/routes.ts 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2892      +/-   ##
==========================================
- Coverage   73.62%   73.30%   -0.32%     
==========================================
  Files         521      523       +2     
  Lines       20937    21092     +155     
  Branches     1844     1841       -3     
==========================================
+ Hits        15415    15462      +47     
- Misses       5512     5620     +108     
  Partials       10       10              
Flag Coverage Δ
api 83.45% <46.56%> (-0.61%) ⬇️
web 61.19% <47.54%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
api/src/api/models/submission.py 90.10% <100.00%> (ø)
api/src/api/resources/survey.py 93.05% <ø> (ø)
api/src/api/schemas/public_upload.py 100.00% <100.00%> (ø)
api/src/api/utils/token_info.py 91.66% <100.00%> (ø)
web/src/apiManager/endpoints/index.ts 100.00% <ø> (ø)
web/src/components/survey/listing/Surveys.tsx 72.17% <ø> (-0.24%) ⬇️
web/src/components/survey/report/SettingsForm.tsx 85.07% <100.00%> (-2.62%) ⬇️
web/src/components/survey/useSurveyLoaderData.ts 100.00% <100.00%> (ø)
api/src/api/services/comment_service.py 65.58% <75.00%> (+0.16%) ⬆️
web/src/routes/routes.ts 87.50% <75.00%> (-12.50%) ⬇️
... and 7 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@NatSquared
NatSquared requested a review from jareth-whitney July 8, 2026 18:38

@jareth-whitney jareth-whitney left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this CHEFS, object storage, and general form cleanup. I figure if we are good stewards of the app then these little improvements come naturally. Nothing blocking, just check out the following:

  • authorization message specificity
  • Number.isNaN vs Number.isFinite or similar
  • Checking that spread of two objects is shallow
  • Console.log (possibly intentional)
  • Token validation vs simple existence check
  • surveyId Number conversion, consider converting once

f'membership.tenant_id:{membership.tenant_id} '
f'user_from_context.tenant_id: {g.tenant_id}')
abort(HTTPStatus.FORBIDDEN, UNAUTHORIZED_MSG)
flask_abort(HTTPStatus.FORBIDDEN, UNAUTHORIZED_MSG)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for giving the authorization service a refresh.

if not self.is_configured():
return {
'status': 'Configuration Issue',
'message': 'accesskey is None or secretkey is None or S3 host is None or formsbucket is None'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering if this error is for the log or returning to the user. We could optionally give users a more generic message.

@classmethod
def get(cls, survey_id):
"""Get survey by the ID."""
"""Get a survey by its ID."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love these little grammar fixes, very much my style.

try {
if (isNaN(Number(submissionId))) {
throw new Error();
if (Number.isNaN(Number(submissionId))) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking if you're specifically looking for NaN or want to check if it's a valid number. You could use something like Number.isFinite() and >= 0 for example.

import { Link } from 'components/common/Navigation';

const FailurePanel = ({ email, handleClose, tryAgain, isInternal }: FailurePanelProps) => {
/* TODO: Populate this with the tenant configuration from the API */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the TODO's, please ignore SonarCloud!

return fetchSurveyReportSettings(String(s.id));
});
const submission = verification?.then((verif) => {
if (!verif?.verification_token) return null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to validate the token or is its existence enough?

@NatSquared NatSquared Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the server response contains a verification_token field, it means that a valid verification resource was found

const { survey } = useSurveyLoaderData();
const isLoggedIn = useAppSelector((state) => state.user.authentication.authenticated);
const surveyLoaderData = useSurveyLoaderData();
const isManagementPath = useIsManagementRoute();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love all of these custom little hooks, very cool.

<Suspense
fallback={
<Button disabled>
{translate('surveySubmit.surveyForm.button.cancel')}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding translation strings here.

// we can still expect the survey data from the nearest route's loader
const fallbackData = useLoaderData();

return (surveyLoaderData ?? publicSurveyLoaderData ?? fallbackData) as SurveyLoaderData;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to fire all these loaders at once? I guess it's a trade off, performance vs resource conservation. You could request loader data 2 if loader data 1 doesn't arrive, for example.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But they're not firing! That's why loaders are so efficient - you can't start them more than once. The loaders are triggered by the user's navigation to the page, and copies of the promise(s) are made available to every call of useLoaderData(). The routes are set up so that no 2 survey loaders can be active at once. Thus, calling useLoaderData() on a route where the loader is not triggered returns undefined and results in 0 XHRs

Comment thread CHANGELOG.MD
- Added a new public document endpoint to the API that generates pre-signed URLs for uploading files to Object Storage. This endpoint takes the file name and content type as input and returns a pre-signed URL that can be used to upload the file directly to Object Storage.
- The new API can also be used to generate pre-signed URLs for downloading files from Object Storage, as well as deleting files from Object Storage.
- Updated the Formio options to include callbacks for handling file uploads, downloads, and deletions. These callbacks use the new API to generate URLs and perform privileged operations on Object Storage, allowing users to upload files during survey submission without needing high-level access to the storage service.
- Cleaned up survey-related component code

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for implementing the Midas touch here!

@sonarqubecloud

sonarqubecloud Bot commented Jul 9, 2026

Copy link
Copy Markdown

@NatSquared
NatSquared merged commit 8685fc4 into main Jul 9, 2026
16 checks passed
@NatSquared
NatSquared deleted the DEP-277-formio-fileupload-backend branch July 9, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants